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ASSTRACT 

To discover factors which contribute to a high 
response rate for questionnaire surveys, the preferences of 150 
college teachers and teaching assistants were studied. Four different. 
cuestionnaire formats using 34 common items were sent to the 
subjects: open-ended; Likert-type (five points, from "strong 
influence to return," to “strong influence not to return); 
checklist; and rank ordering of five to eight factors. Subjects were 
also asked how many questionnaires *hey had received, and returned, 
in the past year. When instructed to choose one of the four 
questionnaire formats and complete it, 54 subjects (36%) completed 
and returned a questionnaire; 22 chose the checklist, 17 the 
Likert-tyoe, 14 the open-ended, and 1 th2 ranking. Characteristics of 
each type of questionnaire which influence rate of return--Ssuch as 
physical characteristics, content, and administration methods--were 
analyzed from the answers received. Subjects preferred clarity and 
brevity; familiar but not overly personel subject matter; and 
convenient methods of completion and return. A humorous followup 
reminder, if any, was preferred. Respondents also wished to know the 
purpose cf the research and the researchers, and *o be assured of 
confidentiality. Non-response was predicted for questionnaires which 
aid no meet these criteria. (GDC) 
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A Study of Factors That Influence Response Rate 


Using a Questionnaire on Questionnaires 


INTRODUCTION 
The use of questionnaires as a research technique has been prevalent 
hi nana’ 04 behavioral sciences because of their low cost, ease of 
administration, large sampling, and ease cf completion. Yet the ques- 
tionnaire has limitations such as low response rate, limited reliability 
and validity information, and impersonalization. Among these limitations, 
response rate seems to be a perennial problem for questionnaire users. 

, While seemingly lacking an underlying theory which would assure a 
high degree of response rate from questionnaire recipients, questionnaire 
users often have to rely on "eimmicks" to attract their recipients to 
respond to fiale qicerioaainae, Many questionnaire advocates have 
painstakingly tried to find tactics that might boost response rates in 
ques treniasee surveys. Factors such as impressive questionnaire features, 
endorgement or sponsorship of an influential organization, assurance of 
sncainity and confidentiality, promises of taken rewards, clever follow-up 
methods, etc., have recurred as research topics. However, a common 
weakness in these tactic-research studies is that only a limited number 
of factors can be studied simultaneously. Information concerning some 
factors is scant while for others abundant. ae to avoid piecemeal 
searching for information concerning the factors for boosting response 


rates, the purpose of the present study was to investigate the most 
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desirable factors, based on preferences of questionnaire users, and to 
provide more detailed information for future questionnaire users. 
Factors which might affect the response rate of a questionnaire 


survey were categorized into three groups: of common characteristics: 


(1) Bhypical features, (2) content, and (3) administration methods. The 
(iadabichaleavtane Goneemnine diene three main factors were as follows: 
(1) What physical features of a questionnaire would influence 
recipients to respond to a questionnaire? 

(2) What kind of questionnaire content would recipients prefer 
or hesitate to answer? 

(3) What methods of administration would induce recipients to 
respond to a questionnaire? 

. An additional research question concerned format preference: Which 
of four questionnaire formats (checklist, open-ended, Likert type, or 
item ranking) do recipients prefer to answer? 

LITERATURE 

A survey of literature was conducted to obtain a list of the most 
commonly used and effective methods of Pa divaieis response rate. There 
were two discouraging aspects to this wedi: First, it appeared that 
the primary signe ‘dt the survey reported in the literature was not the 
analysis of the survey method, but to answer some substantive question. 
The article on the survey was secondary, and in some cases a by-product 
of the original purpose. Second, techniques that significantly increase 
response rate in one survey failed to work in other surveys. A summary 
of some of the possible factors increasing response rate is presented 


in Table i: 


Table 1 


A Survey of Factors and Their Effects 


on Questionnaire Response Rates 


; Will Increase the Will Not Necessarily 
This Technique Response Rate Increase the Response 
, Using colered printed Horowitz & Sedlacek, 
stock 1974; Linsky, 1975 
Enclosing a stamped Erdos, 1957a; 
return envelope Veiga, 1974; 


Linsky, 1975 


A personal, signed Erdos, 1957a; Horowitz & Sedlacek, 
cover letter , Matteson, 1974 1974 
Linsky, 1975 
A researcher of status Erdos, 1957a; Horowitz & Sedlacek, 
and prestigious Linsky & Spendlove, 1974 
sponsorship 1967; 
Carlsmith, et al, 1973 
Use of follow-ups Diliman, et al, 1974; 
Gleason & Huck, 1974; 
Anderson & Berdie, 1975; Pe 
Sketh, 1975 — 
Mailing directly to Rockman, 1973 
the respondent . 
Promise of confiden- Erdos, 1957a Linsky & Spendlove, 
tiality or anonymity 1967; Fuller, 1974 
Including or promising Erdos, 1957b; 
to send a premium or Rockman, 1973; 
monetary incentive Carlsmith, et al, 1973; 
Gleason & Huck, 1974; A 
Linsky, 1975 ‘ 
Short questionnaires Erdos, 1957a; 


Dillman, 1974 


Pre-coded (as opposed Falthzik & Carroll, Linsky, 1975 
to open-ended) questions 1971 : 


METHODOLOGY 

With respect to the questions under investigation, the following 
design and administration was implemented in this study. 
Sampling , 5 

The population of the study was all teaching members of the staff 
of Southern Illinois University in the College of Education and Liberal 
Arts and Sciences in the fall of 1975, including all full-time or part- 
time teaching faculty or teaching assistants. From this population, 150 
subjects were selected randomly from eight randomly selected departments, 
75 from each college. 
Questionnaire Format 

In order to find the format preferred by the sample, four different 
question formats were chosen for comparison: open-ended, Likert-type, 
checklist, and rank order. Items within each format were constructed to 
elicit the degree to which the various questionnaire characteristics 
(physical features, content, and administration) would influence return 
or non-return of a questionnaire. The Likert scale provided five response 
points from "strong influence to return" to "a strong influence NOT to 
return." On the rank order format, the subjects were asked to rank from 
five to eight factors with regard to their influence on return of a 
questionnaire. The overall content of the four question formats were as 
similar as possible, with 34 items in acuiidi aeevind as the basis for the . 
analysis of the research questions one theough threes In addition, two 
questions were included on each form to ask the subjects how many ques- 
tionnaires they had received during the past twelve months, and of these, 


how many they had returned. 


w 


Procedure . 


Each subject received a packet of the four question formats and 
was asked to choose only one to complete and return. The forms were all. 
on the same sized paper and were inserted into the envelope in random 
order. Two weeks were allowed for returning the completed questionnaire. 
RESULTS AND DI SCUSSION 


The response rates by questionnaire are presented in Table 2. 


Table 2 


Response Rates by Questionnaire Type 


Type of Questionnaire Number Percent Percent of 


Returned Returned of 150 No. Returned 
Checklist 22 14.7 40.7 
Likert-type 17 11.3 31.5 
Open-ended 14 9.3 25.9 
Ranking 1 0.7 1.9 e 
OVERALL : 54 36.0 100.0 


The 36% response rate is discouraging (to say the least), but 
since there was no follow-up used, and the questionnaires were sent out 
just betore Thanksgiving break, the researchers were not too surprised, 
in eleva Performing a giti-wauabe sosdiess of fit on the four types 
of questionnaires returned was highly significant (x? = : 179, df = 3, 

p < .001). If the questionnaire type using ranking were eliminated, 
there was. no aigniftcant preference of questionnaire type using the 


Chi-square goodness of fit test. (It was decided to eliminate the one 


ranking questionnaire from further analysis for lack of,a basis of 
comparison. ) 

A discussion of the wording of the questions and the results will 
be presented for each questionnaire format separately. The complete 
results for each of the common items by format of questionnaire is 
presented in Appendix A.) 

Checklist Format 

After a general introduction of the purpose of the summary, the 
respondents were asked to "check as many items that you think are 
importan: and that would influence whether or not you would return a 
questionnaire." For ascii, with respect to the physical appearance 
characteristics, the question was asked, "Which of the following physical 
appearance techniques would influence you to retwn a questionnaire?” 


The seven most important features of influencing return of 


questionnaires is given in Table 3. 


Table 3 
Features Influencing Questionnaire Return 


(from Checklist Format) 


Characteristic N(out of 22) Percent : 
Research purpose is explained 22 100 
Simple, clear directions 21 96 
Stamped, self-addressed return envelope 21 96 
Less than 4 pages | 21 96 
Topic you have experience with 21 _ 96 ; 
Neat overall appearance 18 82 
Letter is addressed to you 18 82 


. 
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Three of the features that were checked least often were 
(1) Promising a token reward for participation - 9%, (2) Answering over 
the phone - 5%, and (3) Using machine answer sheets - 0%. 

The checklist format questionnaire included the following question 
that did not appear as directly on the other formats: "Which of these 
areas of information would you hesitate to answer on a questionnaire?" 
The results of this question, including all areas listed, is given in 


Table 4. : OS 


Table 4 


Areas of Information Causing Hesitation to Respond 


i 


Percent of Percent of 


Number Respondents Responses 

Areas of Information Responding (N=22) (N=41) 
Age 0 0 oF 
Educational status 0 0 0 
Occupation and position 0 o 0 
Savings 14 64 34 
Hobbies and interests 0 0 0 
Opinions 0 0 0 

- Marital status t 5 2 
Name 2 ; 9| 5 
Income | 9 41 22 
Number of children | 0 O° 0 
Personal properties 10 45 24 
Club memberships 5 23 12 


gt 


Obviously, a researcher should expect little response to questions 
related to certain areas of information, even perhaps, if the areas are 


basic to the purpose of the research. | 


Likert-type Format 


After a brief introduction, the questionnaire using Likert items 


. 


asked, "To what extént would each of the following circumstances influence 
whether or not you would complete and return a questionnaire?" The 
respondents were asked to place an X in columns headed "strong Influence 
to Return," "gome Influence to Return," "No Influence," "Some Influence 
Not to Return," and "Strong Influence Not to Return." Those character- 
istics that were most frequently chosen to have strong or some influence 


are presented in Table 5. 


| “a 
Table 5 | 
Factors Influencing Questionnaire Return (from Likert Format) 
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Characteristics N(out of 17) Percent 
Research purpose is explained , 17 100 
Sincere, informal tone of letter 17 100 
Simple, clear directions 17 100 
Staipedls self-addressed return envelope 17 100. 
Researcher personally known s 27 ~ 100 
Answer directly on questionnaire 16 94 


‘Typewritten pages 14 .- 82. 
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Three of the characteristics that were least likely in influence 


returns were (1) Use of Rasiadtandies 18%, (2) Use of taken reward, 35%, 
(3) Mailing questionnaire to respondent's home, 24%. 

There were a few items unique to the Likert-format questionnaire. 
The items are presented in Table 6 with "Strong" and "Some" frequencies 


combined. 


Table 6 
Influences of Items Unique to Likert-format Questionnaire (N=17) 


Influence No Influence NOT 
Characteristic TO Return Influence TO Return 


a a a a a 
Physical appearance 
Cartoon is added on cover letter a 10 | 6 


Cover letter written in friendly manner 15 a: 0 


Cover letter closed with reproduced 


signature a a. 5 
Questions take at least 12 minutes 3 5 8 
Content 
You are familiar with quest. topics 17 0 0 
You oppose questionnaire. topics 10 4 2 
You are required to look up answers 1 4 12 
Administration 


You are given the questionnaire in 


person while shopping 2 "48. ' 


Open-ended Format 


Since responses to open-ended items are dependent upon how the 


question is worded, the open-ended questions are given below. 


In each of the general areas below, what specific factors, if 

any, do you think would influence you to complete a questionnaire 
and return it. Please don't feel restricted by the examples: 

(1) The physical appearance (for example, length, elegance, format), 
(2) The content of the questions (for example, items which are 

too personal, too much trouble), 

(3) Just who the researcher is (for example, his status, connec- 
tions, affiliation, purposes), 

(4) The method of administering the survey (for example, by what 
means you receive the questionnaire, how you are expected to return 
it, any incentive received or promised). 


The responses were categorized into one of those on the Likert or 


checklist formats. The most frequently mentioned characteristics are 


given in Table 7 below. 


« 
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Table 7 


Factors Influencing Questionnaire Return (from Open Format) 


Characteristics | N (out of 14): ae 


confidentiality 13 93 
Less than 4 pages 10 i 71 
. Research purpose is explained 9 64 
Simple, clear directions 8 . 57 


f 
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When responses from all three types of questionnaires are combined, 
the most frequently cited factors influencing questionnaire returns are 


presented in Table 8, “ 


Table 8 
Characteristics of Questionnaires Most Frequently Cited as 


Influencing Return of Questionnaire Across All Three Formats 


Characteristics N Percent (out of 53) 


Physical appearance 
Less than 4 pages Wy 83 


Neat overall appearance 36 68 


Introductory Letter 


Research purpose is explained 48 / 94 
Clear, simple instructions 46 87 
Confidentiality is promised 42 79 
Research results returned: _ 82 60 
Topics covered you have experience with 32 60 


Method of Administration 


Stamped, self-addressed return envelope 42 79 
Mailed to office 28 53 
Answer on questionnaire 33 62 


Methods of Follow-up 


A humorous reminder - , 22 42 
Enclose a second questionnaire a: a 32 
Characteristics of Researcher 


Personally known ‘ 31 58 


Works with respected organization 13 24 45 


‘ 


In response to the question regarding the number of questionnaires 
received and returned during-the past 12 months, the following distri- 


\ 


- butions were generated. 


. 


f Table 9 


Questionnaires Received and Returned during Past 12 Months 


Received Returned 
Number of ———— — 
Questionnaires Frequency Percent ‘Frequency Percent 
0-6 35 66 % 42 719 % 
T =- 22 16 30 11 21 
More than 22 2 \ : 0 0 
MEAN 6.0 hey 
MEDIAN 4.0 3.0 
CONCLUSIONS 


The results indicate that a sample of presumably frequent question- 
naire users, sation placed in a position of respondent, favored a 
questionnaire having characteristics of clarity and brevity; having a 
familiar, yet ne | subject matter; and being convenient to 


administer and return. The majority did not wish to be reminded; but if 


b 


one must, use a humorous. follow-up. They wanted to know the purposes of 
_ the research, to be promised confidentiality, to be informed ofthe 
results, and to know the researcher(s) who possibly had some status. 
The sagan tya dn dase non-response were to create questionnaires which : 
were too long, too. hard to fill out, too personal, had wonetaulitan gibjeck 


ne, 
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matter, and were detiwaned at home with no stamped, self-addressed return 
envelope. Other factors mentioned in this study as techniques frequently 
manipulated by researchers seemed not to possess the power to affect a 

return rate that many studies have indicated. The researchers were 

aware of the obvious limitations of a 36% return ti, and generaliza- os 
bility of the results. However, the finding of the present study 

suggested that certain techniques could be useful in boosting return rate 

and a questionnaire user should take them into consideration before 


conducting a questionnaire survey. 
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“o Appendix A — "7 
CHARACTERISTICS INFLUENCING QUESTIONNAIRE RETURN BY TYPE OF QUESTIONNAIRE 


Checklist Likert Open Combined 
Characteristic AN=22) (N17) (NEL4) (N53) 
PHYSICAL APPEARANCE ee Ef BS 2S 
Typewritten pages 13 59 14 82 0 oO 27 «51 
Printed pages 7 32 13 77 i. 13 21 40 
Smaller or larger (thaa 8 1/2/x 11) 10 46 5 29 0 0 15 28 
Machine answer sheet <4. “0 8 47 a ¢) 8 16 
colored paper 8 36 3 18 1 #7 12 23 
Less than 4 pages 21 96 13 7 LO: FL 448g 
Sponsoring organization on letterhead 12 55 é 14 82 0 oO 26 49 
Neat overall appearance 18 82 16 94 2 14 36 68 
CHARACTERISTICS OF INTRODUCTION LETTER 
Research purpose is explained 22 100 17 100 9 64 48 91 
Letter is addressed to you 18 ‘82 6 35 as | 25 (47 
Confidentiality is promised 14” 64 15 88 13 93 42 79 
- Person to answer is specified 6 27° . Il 65 0 oO 17, 32 
Sincere, informal tone of letter li 50 17 100 0 60 28 53 
Simple, clear directions : 21 96 17 100 6 57 46 87 
Token reward (eg coin) for particip. 2 o° 6 35 0 0 8 15 
Research results promised . : 15 68 14 82 3. 2) 32 60 
Topic you have experience with 21 96 ii. 65 <Q 0 32 60 
METHOD oF ADMINISTRATION F | 
hamuintre delivered to you personally . 8 36 e& 0 oO 8 22 a 
Stamped, self-addressed return envelope 21 96 17 100 4% 29 42 79. 
Delivered by boss be 6 35 0 #60 7. 23 
Picked up by researcher 4 18 3 18 0 oO 7 #13. 
Mailed’ to your home 3 14 Yo die Be 0 0 7 #13 
Mailed to your office ee | ay a 1 g 28 53 
Answer over phone L 6 7 4l oO <F 8 15 
Answer on questionnaire by ie of 16 94 0 oO 33 62 
METHODS OF FOLLOWUP 
e A second. questionnaire 10 46 7 41 0, 0 17 (32 
A telephone reminder 4 18 8 47 L 3 13 25. 
Small token gift with 2nd Q. 3 #14 5 29 0 Oo 8: ES: 
A humorous reminder 12 55. 10 59 0 oO 22 42 
CHARACTERISTICS OF RESEARCHER ' 
Works with respected person/organiz'n 8 36 a2 31 4- 29 24 45 
Located in your community 5 23 lo 59 0 oO -, 415. 28 
Located out of state 3 14 3 18 0 0. °6 ll 
Personally known 14 64 17 100 0 Oo 31 58 
— : 
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